{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy a Serverless Model Server with Nuclio-KFServing\n",
    "  --------------------------------------------------------------------\n",
    "\n",
    "The following notebook demonstrates how to deploy an XGBoost model using nuclio + KFServing (a.k.a <b>Nuclio-serving</b>)\n",
    "\n",
    "#### **notebook how-to's**\n",
    "* Write and test model serving (KFServing like) class in a notebook.\n",
    "* Deploy the model server as a Nuclio-serving function.\n",
    "* Invoke and test the serving function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='top'></a>\n",
    "#### **steps**\n",
    "**[define a new function and its dependencies](#define-function)**<br>\n",
    "**[test the model serving class locally](#test-locally)**<br>\n",
    "**[deploy our serving class using as a serverless function](#deploy)**<br>\n",
    "**[test our model server using HTTP request](#test-model-server)**<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# mlrun: ignore\n",
    "# if the nuclio-jupyter package is not installed run !pip install nuclio-jupyter\n",
    "import nuclio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='define-function'></a>\n",
    "### **define a new function and its dependencies**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%nuclio: setting kind to 'nuclio:serving'\n",
      "%nuclio: setting 'MODEL_CLASS' environment variable\n"
     ]
    }
   ],
   "source": [
    "# specify function metadata\n",
    "%nuclio config kind=\"nuclio:serving\"\n",
    "%nuclio env MODEL_CLASS=XGBoostModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: numpy in /conda/lib/python3.6/site-packages (1.18.1)\n",
      "Requirement already satisfied: xgboost in /conda/lib/python3.6/site-packages (1.0.2)\n",
      "Requirement already satisfied: numpy in /conda/lib/python3.6/site-packages (from xgboost) (1.18.1)\n",
      "Requirement already satisfied: scipy in /conda/lib/python3.6/site-packages (from xgboost) (1.4.1)\n"
     ]
    }
   ],
   "source": [
    "%%nuclio cmd\n",
    "pip install numpy\n",
    "pip install xgboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%nuclio: setting spec.build.baseImage to 'mlrun/mlrun'\n"
     ]
    }
   ],
   "source": [
    "%nuclio config spec.build.baseImage = \"mlrun/mlrun\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import xgboost as xgb\n",
    "from mlrun.runtimes import MLModelServer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class XGBoostModel(MLModelServer):\n",
    "    def load(self):\n",
    "        # this is called once to load the model\n",
    "        # get_model returns file path (copied to local) and extra data dict (of key: DataItem)\n",
    "        # model object can be accessed at self.model_spec (after running .get_model)\n",
    "        model_file, _ = self.get_model(\".bst\")\n",
    "        self._booster = xgb.Booster(model_file=model_file)\n",
    "\n",
    "    def predict(self, body):\n",
    "        try:\n",
    "            # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970\n",
    "            events = np.array(body[\"instances\"])\n",
    "            dmatrix = xgb.DMatrix(events)\n",
    "            result: xgb.DMatrix = self._booster.predict(dmatrix)\n",
    "            return result.tolist()\n",
    "        except Exception as exc:\n",
    "            raise Exception(f\"Failed to predict {exc}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following end-code annotation tells ```nuclio``` to stop parsing the notebook from this cell. _**Please do not remove this cell**_:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# mlrun: end-code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "______________________________________________"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='test-locally'></a>\n",
    "### **test the model serving class locally**\n",
    "The class above can be tested locally. Just instantiate the class, `.load()` will load the model to a local dir.\n",
    "\n",
    "> **Verify there is a `model.bst` file in the model_dir path (generated by the training notebook)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# a valist model.bst file MUST EXIST in the model dir\n",
    "# model_dir = os.path.abspath('./')\n",
    "model_dir = \"/User/mlrun/kfp/032e6d59-6bfe-4ee7-bcf6-1fb26e5db550/1\"  # /model.bst'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_server = XGBoostModel(\"my-model\", model_dir=model_dir)\n",
    "my_server.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We can use the `.predict(body)` method to test the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[22:34:39] WARNING: /workspace/src/learner.cc:979: Number of columns does not match number of features in booster. Columns: 1 Features: 4\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[[0.7845662832260132,\n",
       "  0.02555863931775093,\n",
       "  0.03296218067407608,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901],\n",
       " [0.7845662832260132,\n",
       "  0.02555863931775093,\n",
       "  0.03296218067407608,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901,\n",
       "  0.0224161297082901]]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_server.predict({\"instances\": [[5], [10]]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='deploy'></a>\n",
    "### **deploy our serving class using as a serverless function**\n",
    "in the following section we create a new model serving function which wraps our class , and specify model and other resources.\n",
    "\n",
    "the `models` dict store model names and the associated model **dir** URL (the URL can start with `S3://` and other blob store options), the faster way is to use a shared file volume, we use `.apply(mount_v3io())` to attach a v3io (iguazio data fabric) volume to our function. By default v3io will mount the current user home into the `\\User` function path.\n",
    "\n",
    "**verify the model dir does contain a valid `model.bst` file**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlrun import new_model_server, mount_v3io\n",
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<mlrun.runtimes.function.RemoteRuntime at 0x7f598f33b940>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fn = new_model_server(\n",
    "    \"iris-srv\", models={\"iris_v1\": model_dir}, model_class=\"XGBoostModel\"\n",
    ")\n",
    "\n",
    "# use mount_v3io() for iguazio volumes or mount_pvc() for k8s PVC volumes\n",
    "fn.apply(mount_v3io())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[mlrun] 2020-06-08 22:14:47,883 deploy started\n",
      "[nuclio] 2020-06-08 22:15:31,375 (info) Build complete\n",
      "[nuclio] 2020-06-08 22:15:40,911 (info) Function deploy complete\n",
      "[nuclio] 2020-06-08 22:15:40,917 done creating default-iris-srv, function address: 18.221.106.241:31687\n"
     ]
    }
   ],
   "source": [
    "# deploy as a serverless function\n",
    "# specify dashboard=<url> when deploying to a remote cluster\n",
    "addr = fn.deploy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"test-model-server\"></a>\n",
    "### **test our model server using HTTP request**\n",
    "\n",
    "\n",
    "We invoke our model serving function using test data, the data vector is specified in the `instances` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# KFServing protocol event\n",
    "event_data = {\"instances\": [[5], [10]]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "resp = requests.put(addr + \"/iris_v1/predict\", json=json.dumps(event_data))\n",
    "print(resp.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**[back to top](#top)**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
